Near-eye display system and method

ABSTRACT

A near-eye light field display for use with a head mounted display unit with enhanced resolution and color depth. A display for each eye is connected to one or more actuators to scan each display, increasing the resolution of each display by a factor proportional to the number of scan points utilized. In this way, the resolution of near-eye light field displays is enhanced without increasing the size of the displays.

TECHNICAL FIELD

The disclosed technology relates generally to near-eye displays, andmore particularly, some embodiments relate to near-eye displays withreduced size and improved resolution and color.

DESCRIPTION OF THE RELATED ART

Head-mounted displays (“HMDs”) are generally configured such that one ormore displays are placed directly in front of a person's eyes. HMDs havebeen utilized in various applications, including gaming, simulation, andmilitary uses. Traditionally, HMDs have comprised heads-up displays,wherein the user focuses on the display in front of the eyes, as imagesare traditionally displayed on a two-dimensional (“2D”) surface. Opticsare used to make the display(s) appear farther away than it actually is,in order to allow for a suitable display size to be utilized so close tothe human eye. Despite the use of optics, however, HMDs generally havelow resolution because of trade-offs related to the overall weight andform factor of the HMD, as well as pixel pitch.

BRIEF SUMMARY OF EMBODIMENTS

According to various embodiments of the disclosed technology, animproved near-eye display with enhanced resolution is provided. Thenear-eye display system includes a display component comprising a lightsource array disposed on one or more actuators; an actuator controlcomponent communicatively coupled to the one or more actuators; and aprocessor unit communicatively coupled to the light source array andthat actuator control component, wherein the light source array isconnected with the one or more actuators such that the one or moreactuators are capable of moving the light source array in accordancewith a scan pattern, and the processor unit is configured to synchronizethe illumination of a plurality of pixels of the light source array withthe scan pattern. In some embodiments, more than one display componentsmay be implemented in a HMD.

According to an embodiment of the disclosed technology, a method ofproviding a near-eye light field display with enhanced resolution isprovided. The method includes measuring by one or more focus sensors aneye focus of a user's eyes; determining by a focus correction modules adesired focus based on the measured eye focus; setting a focus of afirst camera and a second camera based on the desired focus; capturingimages within a field of view of the first camera and the second camera;processing the captured images; setting a focus of a first display and asecond display; and providing the captured images to the first displayand a second display; wherein the first display is disposed in front ofa first eye of the user and the second display is disposed in front of asecond eye of the user.

Other features and aspects of the disclosed technology will becomeapparent from the following detailed description, taken in conjunctionwith the accompanying drawings, which illustrate, by way of example, thefeatures in accordance with embodiments of the disclosed technology. Thesummary is not intended to limit the scope of any inventions describedherein, which are defined solely by the claims attached hereto.

BRIEF DESCRIPTION OF THE DRAWINGS

The technology disclosed herein, in accordance with one or more variousembodiments, is described in detail with reference to the followingfigures. The drawings are provided for purposes of illustration only andmerely depict typical or example embodiments of the disclosedtechnology. These drawings are provided to facilitate the reader'sunderstanding of the disclosed technology and shall not be consideredlimiting of the breadth, scope, or applicability thereof. It should benoted that for clarity and ease of illustration these drawings are notnecessarily made to scale.

FIG. 1 is an example diagram illustrating the basic theory of near-eyelight field displays in accordance with embodiments of the technologydescribed herein.

FIG. 2 is an example light field system in accordance with embodimentsof the technology disclosed herein.

FIG. 3 is an diagram illustrating the enhanced resolution capable inaccordance with embodiments of the technology disclosed herein.

FIGS. 4A and 4B illustrate and example scan pattern and enhancedresolution in accordance with embodiments of the technology disclosedherein.

FIG. 5 is an example display configuration having one or more focussensors disposed in between pixels of the display in accordance withembodiments of the technology disclosed herein.

FIG. 6 illustrates an example basic light field system in accordancewith embodiments of the technology disclosed herein.

FIG. 7 is an example process flow in accordance with embodiments of thetechnology disclosed herein.

FIG. 8 illustrates an example computing module that may be used inimplementing various features of embodiments of the disclosedtechnology.

FIG. 9 illustrates an example improved near-eye display system inaccordance with embodiments of the technology disclosed herein.

The figures are not intended to be exhaustive or to limit the inventionto the precise form disclosed. It should be understood that theinvention can be practiced with modification and alteration, and thatthe disclosed technology be limited only by the claims and theequivalents thereof.

DETAILED DESCRIPTION OF THE EMBODIMENTS

As discussed above, HMDs generally employ one or more displays placed infront of the human eye. 2D images are shown on the displays, and the eyefocuses on the display itself. In order to provide a clear, focusedimage, optics placed between the eye and the display make the displayappear farther away than the display may actually be in reality. In thisway, the eye is capable of focusing beyond the space occupied by thedisplay.

HMDs are generally either too large, have limited resolution, or acombination of both. This is due to the distance between pixels in thedisplay, or pixel pitch. When there is sufficient distance between thedisplay and the eye, pixel pitch does not impact resolution to a greatextent, as the space between pixels is not as noticeable. However, HMDsplace displays near the eye, making pixel pitch an important limitingfactor related to resolution. In order to increase resolution, largerdisplays are necessary to increase the number of pixels in the display.Larger displays require larger optics to create the illusion of spacebetween the eye and the display.

Traditionally, the display in a HMD is a 2D surface projecting a 2Dimage to each eye. Some HMDs utilize waveguides in an attempt tosimulate 3D images. Waveguides, however, are complex, requiring precisedesign and manufacture to avoid errors in beam angle and beamdivergence.

One solution that provides true three-dimensional images is the use ofnear-eye light field displays. Similar to light field cameras, anear-eye light field display creates a representation of light as itcrosses a plane that provides information not only relating to theintensity of the light, but also to the direction of the light rays.Traditional 2D displays only provide information regarding the intensityof light on a plane. Some light field HMDs under development utilizewaveguides with diffractive optical elements in an attempt to synthesizelight rays on a plane. However, this approach is complex, requiringprecise design and manufacture to avoid errors in light ray angle anddivergence.

FIG. 1 illustrates the basic operation of a near-eye light field display100 in accordance with embodiments of the technology disclosed herein.As illustrated, the near-eye light field display 100 comprises a lightsource array 110 and an array of lenses 120. Non-limiting examples of alight source array 110 include an LED, OLED, LCD, plasma, laser, orother electronic visual display technology. The array of lenses 120 maycomprise a plurality of lenses, each configured to provide a differentperspective for objects within each lens's field of view. In someembodiments, array of lenses 120 may comprise a plurality ofinjection-molded lenses. By capturing multiple views of the same fieldof view, the direction of the rays of light as they impact the differentsections of the lenses are captured, providing an indication of thelocation and depth of the object within the field of view, representedby the virtual image 130 illustrated in FIG. 1. This virtual image 130represents the actual position of an object in space, which is beyondthe point in space where the light source array 110 is located. Thearray of lenses 120 enables the eye 140 to focus not on the point inspace occupied by the light source 120, but instead to focus on thepoint in space represented by the virtual image 130. In other words, byutilizing a near-eye light field display, the eyes can “look through”the display and focus on a virtual image 130 beyond the light source(display) 110. Note, however, that divergence of the rays of light isprovided by the distance between the light source array 110 and thearray of lenses 120. If the spacing between the light source array 110and the array of lenses 120 is decreased, the divergence of the rays oflight will increase, and the virtual image 130 will appear to be closer.Conversely, if the spacing between the light source array 110 and thearray of lenses 120 is increased, the divergence of the rays of lightwill decrease, and the virtual image 130 will appear to be further away.Accordingly, a light field display may be constructed from a traditionalHMD by providing a means of adjusting the focus distance between thelight source array 110 and the array of lenses 120 according to thedesired apparent distance of the virtual image 130. This adjustment infocus distance changes the direction of the rays as required for a lightfield display. In one embodiment, the focus adjustment is done withinthe retention time of the eye while different portions of the lightsource array are lit up so that different portions of the virtual image130 appear at different distances. In one embodiment, the focusadjustment is done when the focus of the eye changes, as can happen whenthe user looks at objects that are closer or further away.

Embodiments of the technology disclosed herein are directed towardsystems and methods for near-eye light field displays. Moreparticularly, the various embodiments of the technology disclosed hereinrelate to near-eye light field displays providing enhanced resolutionand color depth compared to conventional near-eye light field displays.As will be described in greater detail below, embodiments of thetechnology disclosed herein enable greater resolution and color bylaterally moving a light source array, such as an LED or OLED display,while controlling the intensity of the light source array insynchronization with a scan pattern. Scanning the light source arraywhile modulating the intensity of the pixels reduces the impact of pixelpitch, resulting in increased resolution without the need for largerdisplays or optics. In one embodiment, the intensity modulation of thepixels is achieved by turning the pixels on and off for a time durationthat is dependent on the desired intensity. For example, if higherintensity is desired on a red pixel than on a blue pixel, the red pixelwould be lit up longer than the blue pixel. Moreover, variousembodiments employ a light-field display with enhanced resolution and 3Dimages without the need for complicated waveguides.

By employing embodiments of the systems and methods described below, itis possible to reduce the size and/or enhance the resolution and colorof traditional HMDs, or convert a traditional HMD into a light fielddisplay.

As discussed above, various embodiments of the technology disclosedherein utilize synchronized illumination of the display during scanningwhile moving the light source array to eliminate the space betweenpixels. FIGS. 4A and 4B illustrate scanning of a light source array inaccordance with embodiments of the technology disclosed herein. FIG. 4Aillustrates a light source array comprising four different coloredpixels 410 (red, blue, green, and yellow). In the illustrated embodimentof FIG. 4A, the display is moved in a raster scan pattern 420. As thedisplay is scanned, the light sources are turned on and off in asynchronized pattern. The type of scan pattern utilized in differentembodiments may be taken into account when computing the synchronizedpattern. As illustrated in FIG. 4A, each pixel is translated to beilluminated near each black dot in the pattern 420, essentially turningeach individual pixel into a 4×4 mini-display. Other embodiments mayemploy other scan patterns 420, such as a 3×3 pattern or a 2×2 pattern.In general, the scan pattern can be any desired Lissajous or similarfigure obtained by having a different frequency of repetition (notnecessarily sinusoidal) for the scan in the x axis and the scan in theorthogonal y axis. By scanning every pixel in accordance with the scanpattern 420, each mini 4×4 display overlaps the others, creating avirtual display having full color superposition (i.e., each color isrepresented by each pixel position). This is illustrated in FIG. 4B.Moreover, the number of pixels per inch is increased by a factor of 16,resulting in higher resolution without the need to utilize a largerdisplay having a greater number of pixels per inch. If the displayoriginally has a VGA resolution with 640 by 480 pixels, the scanningconverts it into a display with 2,560 by 1,920 pixels, two times betterthan 1080p resolution. The overall increase in resolution isproportional to the number of scan points included within the scanpattern 420.

FIG. 3 illustrates the benefits of scanning a light source array inaccordance with the example scan pattern of FIGS. 4A and 4B. The Y-axisindicates the intensity of light from a light source, such as a lightsource array, while the X-axis indicates the angle of the light. Angleis shown because the human eye detects intensity as a function of angle.With traditional displays, such as LED displays, as the light ismodulated, the intensity of the light changes as a function of angle indiscrete steps, as illustrated by blocks 310 of FIG. 3. Each steprepresented by blocks 310 represents one pixel in the fixed display. Thefigure shows the case for a display with 100% fill factor where there isno space between pixels. In other embodiments, the display may have afill factor less than 100%, leaving a gap between pixels with zerointensity. The intensity as a function of angle with a fixed display ispixelated, meaning there are discrete steps in intensity in thetransition from one pixel to the next that are of sufficiently lowangular resolution as to be perceivable by the eye. By moving thedisplay, however, the convolution of the pixel pitch and the change inangle of the modulated light produces a smoother curve 320, indicatinggreater resolution. This results in enhanced resolution, as can beappreciated when compared to the desired resolution indicated by 330.Some embodiments may result in near perfect resolution. In variousembodiments, the intensity modulation for each pixel may itself bedigitized rather than continuous, but there is still significantimprovement in resolution compared with the fixed display.

FIG. 9 illustrates an example improved near-eye display system 900 inaccordance with embodiments of the technology disclosed herein. Theimproved near-eye display system 900 may be implemented in a pluralityof different HMD solutions, and could also be used to improvetraditional HMD units to provide greater resolution without the need forlarger displays or optics. As illustrated in FIG. 9, improved near-eyedisplay system 900 includes one or more display components 950. Eachdisplay component includes a light source array 910. Non-limitingexamples of a light source array 910 include an LED, OLED, LCD, plasma,or other electronic visual display technologies. A processor unit 940may be connected to the light source array 910 to control theillumination of the pixels of the light source array 910. In someembodiments the processor unit 940 may include one or more of amicroprocessor, memory, a field programmable gate array (FPGA), and/ordisplay and drive electronics. A light field lens 960 is disposedbetween the display component 950 and the human eye (not pictured).

The light source array 910 may be disposed on one or more actuators 920.In various embodiments, the one or more actuators 920 may include one ormore of: voice coil motors (“VCMs”); shape memory alloy (“SMA”)actuators; piezoelectric actuators; MEMS actuators; a combinationthereof; among others. In some examples, the one or more actuators 920may comprise a MEMS actuator similar to the MEMS actuator disclosed inU.S. patent application Ser. No. 14/630,437, filed Feb. 24, 2015, thedisclosure of which is hereby incorporated herein by reference in itsentirety. Non-limiting examples of material for connecting the lightsource array 910 to the one or more actuators 920 include: epoxy;solder; metal pastes; wire bonding; among others. To control theactuators 920, the improved near-eye display system 900 may includeactuator control components 930, including electronics for controllingthe actuators 920 and sensors identifying the position of the actuators920. In various embodiments, the actuator control components 930 may becommunicatively coupled to the processor unit 940 so that illuminationof the light source array 910 may be synchronized with the movement ofthe one or more actuators 920.

The improved near-eye display system 900 is capable of enhancing spatialresolution of images, i.e., how closely lines can be resolved in animage. In terms of pixels, the greater the number of pixels per inch(“ppi”), the clearer the image that may be resolved. When operated inaccordance with the scan pattern discussed with respect to FIGS. 4A and4B, the resolution of HMDs employing the improved near-eye displaysystem 900 can achieve greater resolution than a traditional display ofthe same size. This enables enhanced HMDs without the need for largerdisplays and optics and, accordingly, larger form factors for the HMDs.

The improved near-eye display system discussed with respect to FIG. 9may be employed in a light field system, such as the example light fieldsystem 200 illustrated in FIG. 2. The light field system 200 is designedto provide an enhanced 3D representation of a scene in a user's field ofview, mimicking transparent eyeglasses. Although discussed with respectto this example embodiment, after reading the description herein it willbecome apparent to one of ordinary skill in the art that the disclosedtechnology can be implemented in any of a number of different HMDapplications.

As discussed above, FIG. 2 illustrates an example light field system 200in accordance with embodiments of the technology disclosed herein. Thelight field system 200 includes one or more cameras 210, a processorunit 220, gyrometers/gyroscopes and/or accelerometers 230, one or moreactuators 240 and actuator control components 250, and a light sourcearray 260. The one or more cameras 210 may be directed to capture auser's field of view. In some embodiments, the one or more cameras 210may be light field cameras, which are designed to capture a light fieldrepresentation of the field of view of the camera (i.e., the light ofthe images are broken up by an array of lenses disposed in front of animage sense to capture both intensity and direction of the light). Otherembodiments may utilize other image sensors as the one or more cameras210, such as traditional video cameras. In such embodiments, the one ormore traditional cameras would capture a series of pictures at differentfocal depths.

The images from the one or more cameras 210 may be fed into processorunit 220 to compute the actual image at different depths based on thecaptured images. For example, where a traditional camera is used, theimages from the one or more cameras 210 may comprise a series ofpictures at different focal depths. To create the three dimensionalactual image, the different captured images are processed to providedepth to the actual image.

To achieve enhanced resolution, the light field system 200 may includeone or more display components 270, similar to the display components950 discussed with respect to FIG. 9. In some embodiments, the displaycomponents 270 may include an array of lenses (not pictured), similar tothe array of lenses 120 discussed with respect to FIG. 1. In otherembodiments, the array of lenses may be disposed between the eye and thedisplay components 270. Various embodiments may include an array oflenses for each eye, i.e., a left eye array and a right eye array.

The captured light field discussed above with respect to the one or morecameras 210 is utilized to compute a sequence for a driver associatedwith the light source array 260, indicating when the plurality of lightsources comprising the light source array 260 should be turned on. Insome embodiments, the light field system 200 may include one or moregyroscopes or accelerometers 230, providing information representativeof the particular position of the user's head. Furthermore, the imagescaptured by the cameras may be processed to determine the motion of theuser's head as well. This information may be fed into the processor unit220 to utilize in computing the light field to account for changes inthe position of the user's head.

To provide depth to the image, one or more actuators for scanning thelight source array in the Z-direction may be included. By scanning inthe Z-direction, the generated light field may be provided at the properfocus. Scanning in the Z-direction may be incorporated into the scanpattern of FIGS. 4A and 4B such that the depth of the image may beprovided to the eye, enabling the eye to focus at a variety of depthsbeyond the surface in space occupied by the light source array. In someembodiments, different colored pixels may be turned on at differentfocal positions, accounting for chromatic aberrations due to thedifference in focal length of each wavelength of the colored pixels.Chromatic aberration may be caused by the human eye, the lens array, orboth. In some embodiments, the array of lenses utilized in the systemmay include doublets, diffraction grating, or other optical techniquesfor eliminating aberrations.

Another benefit of scanning the display is the ability to includedifferent sensors within the display without sacrificing resolution.FIG. 5 illustrates an example display configuration 500 in accordancewith embodiments of the technology disclosed herein. As illustrated, thedisplay configuration 500 includes pixels in three colors (blue, red,and green). Dispersed in between the colored pixels are sensors 510.Each sensor 510 may be configured to scan a small portion of the eye.The scanning by each sensor 510 may follow the same scan pattern asdiscussed above with respect to FIGS. 4A and 4B. The sensors 510 maypick up differences in the light reflected by different portions of theeye, distinguishing between the iris, pupil, and the whites of the eyes.In some embodiments, the sensors 510 may be used to determine the lightreflected from the retina. The light reflected from the retina may beused to determine the focus of they eye. For example, when the eye isfocused at the current position of the display, the light from thedisplay forms a small point on the retina. The reflected light similarlyforms a small dot on the display surface where the sensors are located.In one embodiment, the focus of the eye is determined by measuring thesize and/or position of the reflected spot from the retina on thedisplay surface. In some embodiments, the sensors 510 may distinguishbetween different parts of the eye based on the light reflected orrefracted off the eye. As the sensors are disposed in the space inbetween pixels, the scanning of the display in accordance with the scanpattern allows for the same proportional increase in resolution whilestill including sensors 510 in the display configuration 500.

In order to ensure that images are processed within the persistence ofvision of the human eye, scanning in the three dimensions may need tooccur at high frequency. For example, in some embodiments the focus scan(Z-direction) may be provided by a 25 Hz frequency triangle wave toensure a 50 Hz update rate for the entire image. Accordingly, thelateral scan (X-Y plane) may require a 500 Hz frequency scan in theX-axis and 5 kHz scan in the Y-axis to ensure that the entire image isdisplayed within one depth of focus. In some embodiments, the frequencyof X-Y scanning may be reduced by simplifying the design.

To reduce the complexity associated with three-axis scanning asdescribed above with respect to the light field system of FIG. 2, abasic light field system may be employed, in which only the light fieldfor objects that are in focus is displayed, thereby limiting the needfor Z-direction scanning. FIG. 6 is an example basic light field system600 in accordance with embodiments of the technology disclosed herein.As illustrated in FIG. 6, the basic light field system 600 includes atleast two cameras, a left camera 602 and a right camera 604. Each camerais configured to provide an image stream to a particular eye, left andright. In many embodiments, the left camera 602 and right camera 604 mayinclude a motorized focus such that the focus of each camera 602, 604can be changed dynamically. The focus of each camera 602, 604 may becontrolled by a camera focus control 606. In some embodiments, eachcamera 602, 604 may have an independent camera focus control 606,respectively.

The focus of each camera 602, 604 may be controlled based on the focusof the user's eyes. The basic light field system 600 may include eyefocus sensors 608, 610 disposed within a display in front of the lefteye and right eye, respectively. Each eye focus sensor 608, 610 mayinclude one or more focus sensors in various embodiments. In someembodiments, the eye focus sensors 608, 610 may be disposed in thespaces between the pixels of a left display 612 and a right display 614.The eye focus sensors 608, 610 may be used to determine where a user'seyes are focused. The information from the eye focus sensors 608, 610may be fed into a focus correction module 616. The focus correctionmodule 616 may determine the correct focus based on the point where theuser's eyes are focused, and provide this information to a display focuscontrol 618. The display focus control 618 may provide this informationto the camera focus control 606. The camera focus control 606 mayutilize the focus information from the display focus control 618 to setthe focus of each camera 602, 604. The vision of a user with eye focusproblems (myopia or hyperopia, nearsighted or farsighted) can becorrected by setting the focus of the cameras to a different depth thanthe focus of the display. In some embodiments, the cameras 602, 604 maybe one or more of a light field camera, a standard camera, an infraredcamera, or some other image sensor, or a combination thereof. Forexample, in some embodiments the cameras 602, 604 may comprise astandard camera and an infrared camera, enabling the basic light fieldsystem 600 to provide both a normal view and an infrared view to theuser.

The display focus control 618 may also utilize the desired focus fromthe focus correction module 616 to set the focus of the displays 612,614, to the focus of each eye.

Once the cameras 602, 604 are set to the desired focus, the cameras 602,604 may capture the scene within the field of view of each camera 602,604. The images from each camera 602, 604 may be processed by aprocessor unit 620, 622. As illustrated, each camera 602, 604 has itsown processor unit 620, 622, respectively. In some embodiments, a singleprocessor unit may be employed for both cameras 602, 604. The processorunit 620, 622 may process the images from each camera 602, 604 in asimilar fashion as described above with respect to FIG. 2. For example,the processor unit 620, 622 may compute a light field for use incomputing a sequence to turn on different light sources on each display612, 614 to provide a greater resolution during scanning of the displays612, 614. The images are displayed to each eye via the displays 612,614, respectively, in accordance with the light field sequence computedby the processor unit 620, 622. In some embodiments, this pseudo-lightfield display may be provided without scanning of the displays 612, 614.In such embodiments, although the resolution of the image may not beenhanced, a light field may still be generated and presented to theeyes.

Although illustrated as separate components, aspects of the basic lightfield system 600 may be implemented as in a single component. Forexample, the focus correction 616, the display focus control 618, andthe camera focus control 606 may be implemented in software and executedby a processor, such as processor unit 620, 622.

FIG. 7 illustrates an example process flow 700 in accordance withembodiments of the technology disclosed herein. The process flow 700 isapplicable for embodiments similar to the basic light field system 600discussed with respect to FIG. 6. At 710, one or more focus sensorsmeasure the eye focus of the user. The measurement may be performed byone or more sensors disposed on a display placed in front of each eye insome embodiments. The eye focus sensors may be disposed in the spacebetween pixels of each display in various embodiments, similar to theconfiguration discussed above with respect to FIG. 5.

At 720, a desired focus is determined. The desired focus is determinedbased on the measured eye focus from 710. The desired focus may bedifferent from the eye focus if the user has focus problems. Forexample, if the user has myopia (nearshightedness), the desired focus isfurther away than the measured eye focus. The desired focus may also bedetermined from the position of the eye, such as close if looking down,or the position of the eye with respect to the image, such as the samefocus as a certain object in the scene, or some of other measurement ofthe eye. Based on the desired focus, the camera focus may be set to thedesired focus at 730. In various embodiments, the camera focus may beset equal to the desired focus. In other embodiments, the camera focusmay be set to a focus close to, but not equal to, the desired focus. Insuch embodiments, the camera focus may be set as close as possible basedon the type of camera employed in the embodiment.

At 740, the cameras capture images of objects within the field of viewof the cameras. In some embodiments, the field of view of the camerasmay be larger than the displayed field of view to enable some ability toquickly update the display when there is rapid head movement, withoutthe need for capturing a new image.

At 750, each display is set to the eye focus. In some embodiments, theeye focus is the same as the desired focus. In other embodiments, thedesired focus is derived from the eye focus identified at 710. In someembodiments, the displays may be set to the eye focus before setting thecamera focus at 730, after 730 but before the camera captures images at740, or simultaneous to the actions at 730 and/or 740.

At 760, the images are displayed to each eye. The images are displayedto each eye via the respective display. In some embodiments, the imagesmay be processed by a processor unit prior to being displayed, similarto the processing discussed above with respect to FIG. 6. In someembodiments, the images may be combined with computer generated imagesto generate an augmented reality image.

As used herein, the term component might describe a given unit offunctionality that can be performed in accordance with one or moreembodiments of the technology disclosed herein. As used herein, acomponent might be implemented utilizing any form of hardware, software,or a combination thereof. For example, one or more processors,controllers, ASICs, PLAs, PALs, CPLDs, FPGAs, logical components,software routines or other mechanisms might be implemented to make up acomponent. In implementation, the various components described hereinmight be implemented as discrete components or the functions andfeatures described can be shared in part or in total among one or morecomponents. In other words, as would be apparent to one of ordinaryskill in the art after reading this description, the various featuresand functionality described herein may be implemented in any givenapplication and can be implemented in one or more separate or sharedcomponents in various combinations and permutations. Even though variousfeatures or elements of functionality may be individually described orclaimed as separate components, one of ordinary skill in the art willunderstand that these features and functionality can be shared among oneor more common software and hardware elements, and such descriptionshall not require or imply that separate hardware or software componentsare used to implement such features or functionality.

Where components of the technology are implemented in whole or in partusing software, in one embodiment, these software elements can beimplemented to operate with a computing or processing component capableof carrying out the functionality described with respect thereto. Onesuch example computing component is shown in FIG. 8. Various embodimentsare described in terms of this example-computing component 800. Afterreading this description, it will become apparent to a person skilled inthe relevant art how to implement the technology using other computingcomponents or architectures.

Referring now to FIG. 8, computing component 800 may represent, forexample, computing or processing capabilities found within desktop,laptop and notebook computers; hand-held computing devices (PDA's, smartphones, cell phones, palmtops, etc.); mainframes, supercomputers,workstations or servers; or any other type of special-purpose orgeneral-purpose computing devices as may be desirable or appropriate fora given application or environment. Computing component 800 might alsorepresent computing capabilities embedded within or otherwise availableto a given device. For example, a computing component might be found inother electronic devices such as, for example, digital cameras,navigation systems, cellular telephones, portable computing devices,modems, routers, WAPs, terminals and other electronic devices that mightinclude some form of processing capability.

Computing component 800 might include, for example, one or moreprocessors, controllers, control modules, or other processing devices,such as a processor 804. Processor 804 might be implemented using ageneral-purpose or special-purpose processing engine such as, forexample, a microprocessor, controller, or other control logic. In theillustrated example, processor 804 is connected to a bus 802, althoughany communication medium can be used to facilitate interaction withother components of computing component 800 or to communicateexternally.

Computing component 800 might also include one or more memorycomponents, simply referred to herein as main memory 808. For example,preferably random access memory (RAM) or other dynamic memory, might beused for storing information and instructions to be executed byprocessor 804. Main memory 808 might also be used for storing temporaryvariables or other intermediate information during execution ofinstructions to be executed by processor 804. Computing component 800might likewise include a read only memory (“ROM”) or other staticstorage device coupled to bus 802 for storing static information andinstructions for processor 804.

The computing component 800 might also include one or more various formsof information storage mechanism 810, which might include, for example,a media drive 812 and a storage unit interface 820. The media drive 812might include a drive or other mechanism to support fixed or removablestorage media 814. For example, a hard disk drive, a floppy disk drive,a magnetic tape drive, an optical disk drive, a CD or DVD drive (R orRW), or other removable or fixed media drive might be provided.Accordingly, storage media 814 might include, for example, a hard disk,a floppy disk, magnetic tape, cartridge, optical disk, a CD or DVD, orother fixed or removable medium that is read by, written to or accessedby media drive 812. As these examples illustrate, the storage media 814can include a computer usable storage medium having stored thereincomputer software or data.

In alternative embodiments, information storage mechanism 810 mightinclude other similar instrumentalities for allowing computer programsor other instructions or data to be loaded into computing component 800.Such instrumentalities might include, for example, a fixed or removablestorage unit 822 and an interface 820. Examples of such storage units822 and interfaces 820 can include a program cartridge and cartridgeinterface, a removable memory (for example, a flash memory or otherremovable memory module) and memory slot, a PCMCIA slot and card, andother fixed or removable storage units 822 and interfaces 820 that allowsoftware and data to be transferred from the storage unit 822 tocomputing component 800.

Computing component 800 might also include a communications interface824. Communications interface 824 might be used to allow software anddata to be transferred between computing component 800 and externaldevices. Examples of communications interface 824 might include a modemor softmodem, a network interface (such as an Ethernet, networkinterface card, WiMedia, IEEE 802.XX or other interface), acommunications port (such as for example, a USB port, IR port, RS232port Bluetooth® interface, or other port), or other communicationsinterface. Software and data transferred via communications interface824 might typically be carried on signals, which can be electronic,electromagnetic (which includes optical) or other signals capable ofbeing exchanged by a given communications interface 824. These signalsmight be provided to communications interface 824 via a channel 828.This channel 828 might carry signals and might be implemented using awired or wireless communication medium. Some examples of a channel mightinclude a phone line, a cellular link, an RF link, an optical link, anetwork interface, a local or wide area network, and other wired orwireless communications channels.

In this document, the terms “computer program medium” and “computerusable medium” are used to generally refer to media such as, forexample, memory 808, storage unit 820, media 814, and channel 828. Theseand other various forms of computer program media or computer usablemedia may be involved in carrying one or more sequences of one or moreinstructions to a processing device for execution. Such instructionsembodied on the medium, are generally referred to as “computer programcode” or a “computer program product” (which may be grouped in the formof computer programs or other groupings). When executed, suchinstructions might enable the computing component 800 to performfeatures or functions of the disclosed technology as discussed herein.

While various embodiments of the disclosed technology have beendescribed above, it should be understood that they have been presentedby way of example only, and not of limitation. Likewise, the variousdiagrams may depict an example architectural or other configuration forthe disclosed technology, which is done to aid in understanding thefeatures and functionality that can be included in the disclosedtechnology. The disclosed technology is not restricted to theillustrated example architectures or configurations, but the desiredfeatures can be implemented using a variety of alternative architecturesand configurations. Indeed, it will be apparent to one of skill in theart how alternative functional, logical or physical partitioning andconfigurations can be implemented to implement the desired features ofthe technology disclosed herein. Also, a multitude of differentconstituent module names other than those depicted herein can be appliedto the various partitions. Additionally, with regard to flow diagrams,operational descriptions and method claims, the order in which the stepsare presented herein shall not mandate that various embodiments beimplemented to perform the recited functionality in the same orderunless the context dictates otherwise.

Although the disclosed technology is described above in terms of variousexemplary embodiments and implementations, it should be understood thatthe various features, aspects and functionality described in one or moreof the individual embodiments are not limited in their applicability tothe particular embodiment with which they are described, but instead canbe applied, alone or in various combinations, to one or more of theother embodiments of the disclosed technology, whether or not suchembodiments are described and whether or not such features are presentedas being a part of a described embodiment. Thus, the breadth and scopeof the technology disclosed herein should not be limited by any of theabove-described exemplary embodiments.

Terms and phrases used in this document, and variations thereof, unlessotherwise expressly stated, should be construed as open ended as opposedto limiting. As examples of the foregoing: the term “including” shouldbe read as meaning “including, without limitation” or the like; the term“example” is used to provide exemplary instances of the item indiscussion, not an exhaustive or limiting list thereof; the terms “a” or“an” should be read as meaning “at least one,” “one or more” or thelike; and adjectives such as “conventional,” “traditional,” “normal,”“standard,” “known” and terms of similar meaning should not be construedas limiting the item described to a given time period or to an itemavailable as of a given time, but instead should be read to encompassconventional, traditional, normal, or standard technologies that may beavailable or known now or at any time in the future. Likewise, wherethis document refers to technologies that would be apparent or known toone of ordinary skill in the art, such technologies encompass thoseapparent or known to the skilled artisan now or at any time in thefuture.

The presence of broadening words and phrases such as “one or more,” “atleast,” “but not limited to” or other like phrases in some instancesshall not be read to mean that the narrower case is intended or requiredin instances where such broadening phrases may be absent. The use of theterm “module” does not imply that the components or functionalitydescribed or claimed as part of the module are all configured in acommon package. Indeed, any or all of the various components of amodule, whether control logic or other components, can be combined in asingle package or separately maintained and can further be distributedin multiple groupings or packages or across multiple locations.

Additionally, the various embodiments set forth herein are described interms of exemplary block diagrams, flow charts and other illustrations.As will become apparent to one of ordinary skill in the art afterreading this document, the illustrated embodiments and their variousalternatives can be implemented without confinement to the illustratedexamples. For example, block diagrams and their accompanying descriptionshould not be construed as mandating a particular architecture orconfiguration.

What is claimed is:
 1. A near-eye display system, comprising: a displaycomponent, comprising a light source array disposed on one or moreactuators, the light source array comprising a focus sensor; an actuatorcontrol component communicatively coupled to the one or more actuators;and a processor unit communicatively coupled to the light source arrayand the actuator control component; wherein the focus sensor is disposedon the surface of the display component between at least two pixels of aplurality of pixels of the display component wherein the focus sensormeasures an eye focus of a user's eyes wherein the light source array isconnected with the one or more actuators such that the one or moreactuators are capable of moving the light source array in accordancewith a scan pattern, and the processor unit is configured to synchronizethe illumination of a plurality of pixels of the light source array withthe scan pattern.
 2. The near-eye display system of claim 1, wherein thelight source array comprises one of: an LED; an OLED; an LCD; a plasmadisplay.
 3. The near-eye display system of claim 1, further comprisingone or more lenses disposed in front of the light source array.
 4. Thenear-eye display system of claim 1, wherein the scan pattern comprises araster scan pattern configured to scan the one or more actuatorslaterally.
 5. The near-eye display system of claim 1, wherein the scanpattern results in a Lissajous curve.
 6. The near-eye display system ofclaim 1, further comprising one or more cameras having a field of viewencompassing a portion of a view of a user, and wherein the processor isfurther configured to compute a light field representation.
 7. Thenear-eye display system of claim 6, wherein the one or more camerascomprise one or more light field cameras.
 8. The near-eye display systemof claim 6, wherein the one or more cameras comprises a motorized focus.9. The near-eye display system of claim 6, wherein the scan patterncomprises a raster scan pattern configured to scan the one or moreactuators laterally, and a depth scan pattern configured to scan the oneor more actuators in the Z-axis.
 10. The near-eye display system ofclaim 1, comprising two of the display components, and furthercomprising: a focus correction module communicatively coupled to thefocus sensors of each of the two display components and configured todetermine a desired focus; a display focus control communicativelycoupled to the focus correction module and configured to set a focus ofthe light source array of each of the two display components; and acamera focus control communicatively coupled to the one or more camerasand configured to set a focus of each of the one or more cameras basedon the desired focus.
 11. The near-eye display system of claim 1,wherein the system comprises a head mounted display.
 12. A method,comprising: using one or more focus sensors to measure an eye focus of auser's eyes, wherein each of the one or more focus sensors is on asurface of a first display or a surface of a second display, wherein thefocus sensor is disposed on the surface of the display component betweenat least two pixels of a plurality of pixels of the display componentdetermining by a focus correction module a desired focus based on theeye focus measured using the one or more focus sensors; setting a focusof a first camera and a second camera based on the desired focus;capturing images within a field of view of the first camera and thesecond camera; processing the captured images to generate processedimages; setting a focus of the first display and the second display; andproviding the processed images to the first display and a seconddisplay; wherein the first display is disposed in front of a first eyeof the user and the second display is disposed in front of a second eyeof the user.
 13. The method of claim 12, wherein the first display andthe second display comprise one of: an LED; an OLED; an LCD; a plasmadisplay.
 14. The method of claim 12, wherein processing the capturedimages to generate the processed images comprises computing a lightfield representation of the captured images, and computing a sequencepattern for turning on and off a plurality of light sources disposed onthe first display and the second display.
 15. The method of claim 14,wherein processing the captured images to generate the processed imagesfurther comprising computing a sequence pattern for turning on and off aplurality of light sources disposed on the first display and the seconddisplay.
 16. The method of claim 14, wherein the sequence patterncomprises a raster scan pattern.
 17. The method of claim 12, wherein theone or more cameras comprises a motorized focus and setting the focus ofthe first camera and the second camera comprises setting the motorizedfocus to a capture focus.
 18. The method of claim 17, wherein thecapture focus is equal to the desired focus.
 19. The method of claim 17,wherein the capture focus is different than the desired focus.
 20. Themethod of claim 12, wherein providing the processed images to the firstdisplay and the second display comprises scanning the first display andthe second display via one or more actuators configured to laterallyscan each of the first and second displays.
 21. A method of enhancingimage resolution in a head mounted display comprising a light sourcearray, the method comprising: using a focus sensor between two pixels ona surface of the light source array to measure a focus of an eye;determining a desired focus using the focus of the eye; setting a focusof a head mounted camera using the desired focus; using the head mountedcamera to capture images within a field of view of the head mountedcamera; processing the captured images to generate processed images,wherein the processed images are generated using one or more of:illumination control for the light source array; focal depths associatedwith the captured images; and a sequence for a driver associated withthe light source array; setting a focus of the display using the focusof the eye; and providing the processed images using the display.